Learning from High Dimensional fMRI Data using Random Projections

The term “the Curse of Dimensionality” refers to the difficulty of organizing and applying machine learning to data in a very high dimensional space. The reason for this difficulty is that as the dimensionality increases, the volume between different training examples increases rapidly and the data becomes sparse and difficult to classify. So, the predictive power of a machine learning algorithm decreases as the dimensionality increases with a fixed number of training examples, which is known as the Hughes effect. One way of dealing with the curse of dimensionality is by projecting data into a lower-dimensional subspace. The statistically optimal way to do this (assuming the data is on or near a linear subspace) is PCA, which projects the data into a subspace that preserves as much of the variation as possible. However, PCA is computationally expensive for high-dimensional data compared to the method of dimensionality reduction through random projections. In fact, the distortion of the data when it is compressed via random projections can be bound by the JL lemma. However, empirical testing of this technique will demonstrate how well it performs in practical machine learning problems. The major benefit of using random projections are that they are a computationally less expensive than PCA particulary when the dimensionality of the data becomes too large for matrix diagonalization to be feasible, as is the case for very high-dimesionality data such as fMRI. The main focus of this paper is applying machine learning algorithms to classify fMRI data and attempting to empirically and theoretically predict the feasibility of applying random projections to the supervised classifying of fMRI data. To that end, we first give a review of some of the applicable theory for random projections, then we empirically examine how logistic regression classification error varies when the data is compressed via random projections. We then explain a theoretical method for estimating an assymptotic bound on the generalization error, estimated using k-fold cross validation.